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NATIONAL DATA NEEDS FOR VOCATIONAL EDUCATION 



Obtaining accurate* comprehensive data for vocional education has been one of 
the naorc elusive objectives of federal policy. For more than two decades, Congress has 
periodically expressed dissatisfaction with the condition of vocational education data and 
sought a variety ol" '•fixes" that, for one reason or anoAa:» have failed to produce better in- 
fonnation. As reauthorization of the Cad D. Perkins Vocational Educadoi Act approaches, 
there are once again calls fcxr better data on vocational education* In some instances, these 
are simply requests for rather specific lands of information — ^for example, participation 
patterns of students with special needs* In other cases, full-blown, comprehensive, na- 
tional data systems have been suggested* 

Given the long history of unsuccessful attempts to in:q)rDve the quality and scope of 
data on vocational education, it is important to understand the lessons of the past before 
embarking on a new data collection effort It is possible to design an effective strategy for 
obtaining accurate information on the vocational education entciprise, but only if many of 
the past mistakes can be avoided* Many misunderstandings about the vocational education 
enterprise and its ability to generate useful national data persist Unless these can be recti- 
fied, reauthorization once again risks asking for the impossible* The inevitable frustration 
when the impossible cannot be achieved will further undermine the credibility of the field. 

This paper seeks to improve future efforts to collect and report data on vocational 
education. It begins by reviewing the history of previous efforts to collect national 
information <mi vocational education, and then examines why collecting accurate, consistent 
data has proven so difficult* The scific kinds of information that arc needed for national 
policy and accountability are discussed next. The five areas involved arc governance, 
student information, staffing, finance, and facilities and equipment. Effective strategies arc 
proposed for collecting this information for each arca. Finally, this paper concludes with 
some recommendations for federal policy* 
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THE HISTORY OF EFFORTS TO COLLECT 
NATIONAL DATA ON VOCATIONAL EDUCATION 



Since the ppssage of the Smith-Hughes Act in 1917, the federal government has 
collected some fonn of statistics on vocational education. Until 1963, tiiese amounted only 
to gross counts of students enrolled in die major program areas supported by federal aid— 
for example, agriculture, business, and trade and industry. The Vocational Education Act 
of 1963 and the 1968 Amendments began requiring a series of annual reports that contained 
oaore detailed reporting. In addition to enrollment in major program areas, states were 
asked to report information on total vocational education enroUment by race. They were 
also asked for information on enrollment of disadvantaged and handicapped students; en- 
rollmenis in secondary, postsecondary, and adult vocational education programs; and ex- 
penditures from federal allocations, as well as state and local expenditures for vocational 
education. 

Respmsibility for collecting this data lay witii the Bureau of Occupational and Adult 
Education (BOAE) in die tiien U.S. Office of Education. Although many states used some 
of their federal vocational education research funds to develop new automated information 
systems, littie uniformity existed across states. Problems witii inaccurate, inconsistent data 
persisted throughout die 1960s, and Congress frequendy complained about the quality of 
informatim on vocational education. 



Project Baseline 

To help address this problem. Congress established Project Baseline in 1970, a 
joint effort of the National Advisory Council on Vocational Education and the U.S. Office 
of Education. In Project Baseline's own words, die project "was established to get die in- 
formation everyone needs and no one seems to have" (Lee, 1972). Project Baseline at- 
tempted to work witfi each of die states to develop a more complete and more reliable na- 
tional picture of sec(Midary and postsecondary vocational education. 

Despite five years of effort. Project Baseline was unable to overcome most of die 
problems diat had plagued die collection of vocational education data diroughout die 1960s. 
Enrollment data still fluctuated wildly from year to year. Follow-up information was either 
unavailable or based on such small response rates diat die data was useless. For example, 
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in 1972, twelve states reported secmdary enrollment in vocational education that was more 
than twenty percent greater than enrollment in 1971. Thirteen states in 1973 and 1974, 
fourteen states in 1975, and seven states in 1976 reported secondary enrollment increases 
of more than twenty percent from the previous year. During the period of 1971 to 1976, 
from twelve to twenty-four states regularly reported annual data on the number of program 
completo^ that represented increases of more than twenty percent over the previous year 
(Benson, Hoachlander, & Johnson, 1980). While general secondary enrollment was still 
growing modestly in the early 1970s, the growth was not large enough to make credible 
changes of more than twenty percent in vocational education enrollments. 

To its credit. Project Baseline did not mindlessly report these lai^e fluctuations. It 
sought explanations from the states, ard, in most instances, thm were good reasons for 
the annual differences. New definitions, alterations in reporting systems, duplicative 
counting, late reporting, and missing data were but a few of the causes of inconsistent 
reporting over time. However, while Project Baseline could explain the problems, it could 
not correct them. The sorry state of vocational education data was left unchanged, and 
Project Baseline ceased coUecting data after 1976. 

The Vocational Education Data System (VEDS) 

In 1976, Congress decided to move primary responsibility for vocational education 
data collection out of BOAE and lodge it with the National Center for Education Statistics 
(NCES). As NCES was the primary repository for educational data and had the expertise 
to oversee the design and implementation of large scale data collection efforts. Congress 
hoped that NCES would produce more reliable information on vocational education. Thus, 
the 1976 Amendments to the Vocational Education Act directed NCES to design and oper- 
ate a Vocational Education Data System (VEDS). Congress wanted VEDS to supply an- 
nual information on vocational education students, programs, program outcomes, staff, 
facilities, and expenditures. Congressman Carl D. Perkins summarized the general pur- 
pose of VEDS in his opening remarks during hearings on December 10, 1981 : "In mandat- 
ing this system. Congress was responding to a lack of adequate data to judge program ef- 
fectiveness and to make important decisions about future directions." 

After almost two years in the design stages, VEDS began collecting data in 1978- 
79. No one knows how much VEDS cost — some estimates were as high as two hundred 



million dollars when the resources expended by state and local personnel were counted— 
but by 1983, the realization was growing that the latest attempt to improve vocational 
education data had failed In 1983, NCES perfomied a number of validity tests on the four 
yean of accumulated data. These checks confumed what many critics of VEDS had argued 
fiom the outset — ^three major problems plagued the system: 

1. Lack of comparabUity among states. 

Data was not comparable from state to state and, therefore, yi-slded misleading 
national totals when aggregated. 

2. Year-to-year variabiUty. 

The data exhibited excessive variation over time, which was difficult to explain, 
Consequentiy, the VEDS data could not be used to describe d«nds over time 
accurately. 

3. Within-state discrepancies. 

When VEDS data from some states was compared to state data fit)m other sources, 
many large discrepancies were observed; these could not be adequately explained. 

Following these findings by NCES, the Office of Management and Budget (0MB), 
in December, 1983, witiidrew approval for collecting VEDS data for 1983 -84 and 1984- 
85. In an unpublished memo to the NCES, the 0MB stated that this was because "VEDS 
has substantial and continuing problems collecting data which are accurate and meaning- 
ful." 0MB directed that the "collection should not be undertaken at this juncture and 
should remain suspended until the Department [of Education] has fully implemented the 
plan for improving VEDS." 

Improving VEDS, however, proved impossible. Although some parts of the sys- 
tem probably did produce reasonably accurate information, the widely circulated horror 
stories (e.g., more Native American secondary students enrolled in vocational education in 
one state than the state's total population of Native Americans) subjected VEDS to such 
ridicule that there was little hope of salvaging any credibility for an improved system. In 
February, 1984, the NCES Administrator, with the concurrence of the Office of Vocational 
and Adult Education (OVAE, formerly BOAE), issued the following statement in a memo 
to the Secretary of Education: 

The current [VEDS] system has serious technical problems. NCES 
believes that the technical problems do not lend themselves to 
correction without a major system redesign. The data arc unreliable 
and subject to serious misinterpretation. 
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Consequently, VEDS was pennanently tenninated, and another major effort to pnxluce ac- 
curate, reliable national data about vocational education had failed. (For an excellent 
analysis of the problems inherent in the design and implementation of VEDS, see Robert E. 
Barnes, "Why VEDS Failed," Office of Planning, Budget, and Evaluation, U.S, 
Department of Education, February, 1984, unpublished paper.) 



Data Collection Under the Perkins Act 

In the Fall of 1984, Congress tried once again. The Carl D. Perkins Vocational 
Education Act directed NCES to develop **a national vocational education data reporting and 
accounting system using uniform definitions." The system required by this section was to 
include the following information in regards to vocational education: (1) students 
(including information concerning race, sex, and handicapping condition), (2) programs, 
(3) program completers and leavers, (4) placement and follow-up, (5) staff, (6) facilities, 
and (7) expenditures in relation to the principal purposes of this Act.** This data collection 
mandate differed from VEDS in one very important respect. NCES was to rely on sample 
surveys rather than a universal census. Furthermore, the information would be updated 
every otiier year ratiier than annually. There was one msgor exception to die use of sample 
surveys. Section 423 directed NCES to collect information on all secondary handicapped 
students enrolled in vocational education and to report tiiis information by four-digit CIP 
code, instructional setting, and handicapping condition.^ 

The Peridns Act also continued the National Occupational Information Coordinating 
Committee and charged it with developing and implementing an occupational information 
system i:o meet the needs of boch vocational education and employment training programs. 
The primary objective of this system was to gather information on labor market supply and 
demand. 

Efforts to comply with these directives on data collection have proceeded much 
more cautiously tiian die earlier attempt witii VEDS. To date, ratiier tiian develop an inde* 

^The CIP code is a two, four, or six digit code based on the Classification of Instructional Programs 
developed by NCES to descrit)e education offerings at the secondary and postsecondary levels. For example, 
CIP Code 08 lepresents all Marketing and Distribution programs. Code 08.04 represents Financial Services 
Marketing, and Code 08.0403 is Credit Mariceting. Thus, the level of specificity increases as the code 
moves from two to four to six digits. CIP codes include academic as well as vocational education offerings 
(Malitz, 1981). 
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pendent system for vocational education data coUecticMi, NCES has relied on a strategy that 
emphasizes using a variety of existing data collection efforts and, when necessary, 
modifying them to obtain better infonnation on vocational education These studies include 
the National Longitudinal Study of the Senior Qass of 1972 (NUi-72), High School and 
Beyond, and the National Education Longitudinal Study of 1988 (NELS-88). NLS-72 and 
High School & Beyond figured prominently in the analyses undertaken by the National 
Assessment of Vocational Education (NAVE), the assessment required of the Department 
of Education by the Pferidns Act (Section 403 (a)), Additioiudly, NAVE, in cooperation 
with NCES and the Office of Special Education and Rehabilitation Services (OSERS), led 
an effort to use the National Assessment of Educational Progress (NAEP) to collect recent 
secondary school transcripts to analyze participation patterns in secondary vocational 
education. 

Special attention was paid to using this NAEP supplement to develop information 
on the course-taking patterns of handicapped students. Although the Perkins Act directed 
NCES to collect information on all handicapped students, NCES maintained that this re- 
quest was unworkable. For reasons that will be made clearer below. Section 423 would 
simply have forced NCES to repeat the VEDS fiasco, even though it would be limited only 
to handicapped students. In contrast, the NAEP transcript supplement, while limited to a 
national sample of handicapped students, has yielded for the first time a rich data base on 
the participation patterns of handicapped students in all aspects of the secondary school 
curriculum, academic as well as vocational, by instructional setting and handicapping 
condition. 

Additional information on secondary vocational education will be available from the 
Schools and Staffing Survey, which was conducted by NCES during the 1987-88 school 
year. The NELS-88, the most recent longitudinal study undertaken by NCES, will also 
provide a longitudinal ')ortrait of participation in vocational education beginning with the 
eighth grade. 

At the postsecondary level, NCES has sought to revamp the Higher Education 
General Information System (HEGIS) to include the full array of institutions offering vo- 
cational education outside of high schools. This revised data collection, called the 
Integrated Postsecondazy Education System (IPEDS), replaces HEGIS and will be a major 
source of data on postsecondary vocational education enroUment and completion, as well as 
postsecondary staffing, finances, and institutional characteristics. Additionally, the post- 



secondaiy transcripts collected for NLS-72 and High School & Beyond provide a wealth of 
data on postsecondary c(»irse-taking patterns in vocational education. Finally, the National 
Pdstsecondary Student Aid Survey (NPSAS) noakes data available on financial assistance 
to postsecondary students participating in vocational education in the fiill array of postsec- 
(MMlary institutions, including pnqnrietary schools* 

NLS-72 and High School & Beyond, as well as the longitudinal study aiaintaincd 
by die U.S. DqMutment of Labor (MLS Youth), have also proven to be excellent sources of 
data for analyzing labor maricet outcomes associated with participation in vocational 
education. Not only do these studies overccnne the problem of poor response rates, but 
they also permit tracking the labor market experiences of participants over much longer 
periods of time. 

In short, much of what Congress requested in the Peridns Act is now available in 
noore useful, accurate, and consistent forms than ever before. Three problems, however, 
remain. First, there are some major information gaps, as well as shortcomings, in the 
existing data collection efforts. For exairf ^s, ..aonnation on state and local expenditures 
for vocational education is virtually absent. No information on facilities has been 
systematically collected since the National Study of Vocational Education Systems and 
Facilities by Woodruf 4^ in 1978. Additionally, the saxttplt sizes of some nf th^^ data systems 
inhibit subnational comparisons among regions and among states. Saiaple size also limits 
the degree of progranunatic detail that can be achieved in analyzing participation patterns 
and progiam outcomes. 

Second, the timing of existing surveys does not always coincide with the cycle of 
reauthorization of the fedf^ral law for vocational education. For example, there will be vir* 
tually no data available for reauthwization on secondary staffing, although the Schools and 
Staffing Survey will eventually yield some useful information on secondary vocational ed- 
ucation teachers. 

Ihird, while the cunent decentralized approach lO data collection is more sensible 
than any of the previous strategies for collecting data on vocational education, it increases 
the risk that important opportunities to inr^nrove information will be missed. Presently, no 
one familiar with vocational education is responsible for helping to coordinate the collection 
of data on vocational education among depaitments of the federal government Education, 
Labor, Commerce (Census), and Defense all have data collection efforts in place that 



prcduce infonnation on vocational education and employment training* While some 
conflicts and inconsistencies among these various data systems are inevitable, charging 
scHne entity familiar with vocational education with the responsibility for minimizing 
unnecessary problems would be useful* A similar need exists within departments and even 
within agencies* Within the Department of Education* for exa'^ple, NCES, OSERS, and 
OVAE all collect data on vocational educaticHi without any clear means for coordination* 
Even within NCES, data collection responsibilities are shared among several branches that 
could benefit fnnn better coordination and communication about what kinds of data on 
vocational education are needed and with what fitquency* 

In sum, throughout the history of Congressional mandates, it has been assumed 
that uniformity in the collection of vocational education data can be obtained from an enter- 
prise whose merits are often imbedded in its diversity. Moreover, legislation has not been 
uequately sensitive to how vocatimal educatim interacts with the rest of the educational 
system and the implications of these interactims for data coUection. As reauthorization of 
the Pericins Act approaches, what is needed in the area of data for vocational education is 
not a mandate for a new, independent, centralized vocational educatim data system* Much 
more desirable would be a process for identifying major gaps in information and for 
deciding who is in the best position to fill them. Additionally, information on vocational 
education can be improved by concentrating on fine-tuning the existing and planned data 
collection effort to improve timing, eliminate inconsistencies, and to take advantage of low- 
cost opportunities to collect additional data. Finally, while data collection should remain 
decentralized, it would be desirable to centralize a corpus of expertise capable of using 
these various data sets for analyzing vocational education and for providing quick, efficient 
responses to requests for information* Before describing this strategy in more detail, it is 
useful to better understand some of the issues and problems that have undermined previous 
efforts to collect data on vocational education. 
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WKY HAS GOOD DATA ON VOCATIONAL EDUCATION 
BEEN SO HARD TO OBTAIN? 



Historically, Congress has sought answers to four seemingly simple questions 
about vocaticxial education: 

d] Who is served? 

(2> What is offered? 

(3) What is accomplished? 

(4) What does it cost? 



As described in the previous sections, answering these questions has proven exceedingly 
difficult Why is this so? 



Who is Served and What is Offered? 

Traditionally, efforts to determine who is served by vocational education have con- 
centrated on attempting to count the number of students enrolled in vocational education 
during the course of tiie school year. Typically, these efforts have also attempted to report 
enrollments in different types of vocational education programs (e.g., agriculture, busi- 
ness, health) with different degrees of derail. Rather than simply reporting enrollment in 
agticulture programs, . jr example, more detail is often desired such as whether the enroll- 
ment is in agricultural science or omanoental horticulture. Moreover, policymakers have in- 
sisted on **unduplicated" counts of students, counting a student only once regardless of 
how many vocational courses a student takes in the course of a year and assigning a student 
to only one vocational education program. As has been repeatedly demonstrated over the 
past twenty-five years of att^^^^pts to collect data in tiiis fashion, this desire for annual, 
unduplicated enrollment by type of vocational program is impossible to obtain. It is impor- 
tant to understand why. 

Part of the problem is definition. Accurately and consistently reporting who is 
served and what vocational education is offered depends first on a clear definition of what 
constitutes vocational education and then on a uniform system for classifying program 
offerings. The Carl D. Perkins Vocational Education Act, Section 521 (3)), defines voca- 
tional education as the following: 



organized educational programs which are directly related to the 
preparation of individuals in paid or unpaid employment in such 
fields as agriculture, business occupations, hotoc economics, health 
occupations, marketing and distributive occupations, technical and 
emerging occupations, modem industrial and agriculture arts, and 
trades and industrial occupations, or for additional preparation for a 
career in such fields, and in other occupations, requiring otho- than a 
baccalaureate or advanced degree and vocaticmal student organiza- 
tioti activities as an integral part of the prpgraia 

States have had considerable latitude in interpreting this defiiution, and, especially at the 
secondary level, there is substantial variation among states in what constitutes vocational 
education. 

For example, although federal law specifically includes industrial arts in vocational 
education, it also defines vocational education as programs ''direcdy related to the prepara- 
tion of individuals in paid or unpaid employment** There is a longstanding debate in sev- 
eral states over whether industrial arts is appropriately termed vocational education because 
these programs often hi ^ not intended to prepare students for employment Industrial arts 
was originally conceived by its proponents as an integral part of general educaticm — expos- 
ing students (mosdy boys, until recentiy) to the "culture of the industrial world" Such ex- 
posure was considered as valuable for the prospective banker or physician as for the 
prospective carpenter or electrician. Some states have maintained this traditicmal conception 
of industrial arts and believe, therefore, that it is not properly included in vocational 
education. 

In otiier states, industrial arts classes serve as introductory courses for further work 
in trade and industrial vocational education. In such states, '^Woodshop," an industrial arts 
course, is no different from '"Woodworking I," a vocational education course. Indeed, in 
TDOTt rural areas with smaller schools, industrial arts may, out of nuxssity, play this intro- 
ductory role because school3 are not large enough to support both industrial arts and sepa- 
rate introductory courses in vocational education. Similarly, in states where high schools 
consist only of grades ten through twelve, industrial arts is more likely to resemble intro- 
ductory vocational education than in states where high schools include grade nine or gra.ics 
seven and eight 

Industrial arts, tiierefore, can mean very different tilings in different states, depend- 
ing upon how a state conceives die purpose of the program and the extent to which pro- 
gram offerings are constrained by geography or the orga^iization of secondary education. 
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Quite prop^ly, these are state decisions made with an eye to local conditions and an un- 
derstanding as to how vocational education relates to the goals and philosophies of the 
state's larger educational enterprise. To insist that states uniformly include or exclude in- 
dustrial arts in vocational education would superimpose a rather mindless federal policy on 
more carefully thought out state policies for no apparrot reason other than the desire for the 
illusion of ''comparable" data. 

It might be argued that one way out of this dilemma would be to instruct states to 
include industrial arts as pan of vocational education only if it serves as the first course in a 
planned sequence of vocational education courses, rather than as a general introduction for 
all students to the world of industry. The difficulty with this approach, however, is that 
industrial aits courses may serve both purposes, that is, an intnxluction for more advanced 
training for those studmts electing to pursue it and a cme-time exposure to industrial culture 
for students pursuing other education careers. Who, then, should be counted as being 
served by vocational education, and if it is only those students pursuing moir advanced vo- 
cational education, how will this be detemiined prospectively? 

Such problems oe not limited to industrial arts. Consider students taking 'Typing 
I," which is typicaMy offered in the business departments of most secondary schools. 
Typing I is part of at least three different vocational education prograiits —Business Data 
Processing, Secretarial Training, and Typing and General Office. Each of these three pro- 
grams constitutes a unique four digit code in the Qassification of Instructional Programs 
(CIP). Moreover, Typing I is also taken by large numbers of students who will not pursue 
any additional vocational education at all. As mere is rarely any data available on students' 
long-range intentions (if indeed students clearly know these intentions), accurately 
assigning students enrolled in Typing I to a specific vocational education program is 
impossible.^ 

Similar problems exist throughout the vocational cuniculuoL Agricultural Science 
and Agricultural Mechanics arc required courses for a variety of different agriculturd pro- 
grams. Health is the beginning course in most health projp'ams. Carpentry is the first 

^This problem is not solved by asking students their intentions or asking them whether they consider 
themselves Nocationa]** students. Experience with past surveys indicates large discrepancies between what 
students say they intend to do in their educational careers and what they actually do. Additionally, when 
asked what type of program they are enrolled in. there are also large discrepancies between the percentage of 
st^jdents who say they are vocational and the percentage of students who actually pursue an organized 
pr)gram of vocational education. 
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course in several different woodworking programs, and Electricity is itquired for electron- 
ics programs and electrician programs* 

In short, at any given point in time, students enroll in courses, not programs. 
Accurately assigning students to a specific vocational program cannot, therefore, be done 
prospectively. Rather, accurate assignment requires letiospective information on the 
courses students have taken. Moreover, as a program often cannot be uniquely identified 
until enrollment in the terminal course of a sequence, accurate assignment to a four or six 
digit program code is most accurately acconq)lished at the time of program conopletion. 

The need for retrospective data to accurately determine who is served by what types 
of vocational programs has clear implications for data collection. The best source of retro- 
spective data, at both the secondary and postsecondary levels, is student transcripts. 
Transcripts not only provide the complete course-taking history that makes possible accu- 
rate assignment to programs, they also offer several additional advantages. With tran- 
scripts it is possible to impose alternative uniform definitions of vocational education and 
assess the consequences. One can easily ask. What difference does the inclusion or exclu- 
sion of industrial arts courses make in the number of students participating in vocational 
education? One may also define more unifomily the types of courses constituting a particu- 
lar vocational education program to ensure that the course content of particular programs is 
defined consistently. Additionally, it is possible to move beyond simple notions of partici- 
pation and examine how much vocational education students take, in what areas, and in 
what sequences. It is also possible to examine how vocational course-taking interacts with 
course-taking in other parts of the secondary and postsecondary cuiricula. 

Transcript studies have proven exceptionally useful for analyzing secondary and 
postsecondary participation patterns in vocational education. Recent studies have been 
sponsored by NCES (National Center for Education Statistics), NAVE (National 
Assessment of Vocational Education), and OSERS (Office of Special Education and 
Rehabilitation Services), and some of the earliest explorations of transcript data were 
conducted by the former National Center for Research in Vocational Education located at 
Ohio State University (Campbell, Orth, & Seitz, 1981; Hoachlandcr & Choy, 1986; Tuma, 
Gifford, & Hoachlander, 1989). As valuable as these studies have been, they suffer from 
several shortcomings that should be alleviated in the future. 

First, none of the efforts to collect transcript data were undertaken with vocational 
education specifically in mind. Even the recent NAEP (National Assessment of 
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Educational Ptogrcss) transcript data collection was undertaken as an afterUiought, and, as 
a consequence, many of the transcript records cannot be linked back to earlier data collected 
by NAEP cm students and schools. Similarly, die two NCES longitudinal studies, NLS-72 
and High School & Beyond, while rich sources of data, are not as useful as they might 
otiicrwise have been. It is not possible, for example, to determine what courses were taken 
at area vocational schools versus diose taken in comprdiensive high schools. Additionally, 
die time between longitudinal studies tends to be eight to ten years, and it is desirable to 
monitor course-taking more frequently.^ What is needed, tiierefore, is a conmiitment to 
routine, a periodic collection of transcripts using a vehicle tailored to the needs of 
vocational education. Vocational education data need not, indeed should not, be die sde 
objective of this transcript collection, but die collection should adequately serve die needs 
for vocational education data. 

Second, the sample size of transcript studies is not always large enough to permit 
sufficiendy detailed analysis of course-taking patterns. The NLS Youdi cohorts, for ex- 
ample, contain about twelve-hundred students in each cohort There are, dieiefore, not 
enough observations to do any detailed analysis of how participation by race, sex, or 
special need varies among different types of vocational education programs. Even High 
School & Beyond, with a sample size of roughly twelve-thousand transcripts, will permit 
detailed analysis of only about thirty of die largest vocational educatiai programs. 

Increasing sample size, of course, increases costs, especially if the transcript data is 
to be linked to information on smdent, school, and parental characteristics and to follow-up 
data on family formation, labor market experience, or further education and training. It is 
useful, therefore, to consider transcript studies of different sizes to serve different 
purposes. For example, an examination of how course-taking patterns differ among states 
or among major racial and ethnic groups might be done with a large sample of transcripts 
that simultaneously collects only a bare minimum of additional data on selected student 
characteristics. Its purpose would be largely descriptive and could be repeated perhaps 
once every five years to analyze major trends over time. In contrast, causal stodies of what 
influences students' course-taking patterns would rely on smaller samples but collect much 



^In fact, one motivation for undertaking the NAEP transcript study was the recognition that the secondary 
transcripts available from High School & Beyond were those of 1982 high school seniors. In the absence 
of more recent data with which to analyze possible changes in participation in vocational education, the 
1982 transcripts would have lost much of their value for reauthorization of the Peikins Act in 1989. 
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more detailed infonnation on characteristics of students, teachers, parents, and schools. 
Such studies would resemble more closely the longitudinal studies conducted by NCES 
and the Department of Labor, but would pay closer attention to the kinds of data necessary 
to untangle course-taking patterns in vocational education. 

Finally, sample sizes of transcript studies are never likely to be large enough for 
detailed analysis at the state or substate level. Nor are they likely to be sufficient for ac- 
countability, as opposed to research and analysis. Therefore, to the extent that counts of 
students served are desired at the state level, it would be more effisctive to concentrate on 
counts of students completing vocational education programs rather dian students enrolled. 
For the reasons noted above, it is much easier to identify unambiguously who has 
conopleted a particular vocational education program than it is to determine who is enroUed 
in one. Moreover, insofar as completioi represents readiness to woric in a field related to 
training, completers are a much more useful indicator of labor market supply than are 
enrollments. 

What degree of programmatic detail is necessary for understandmg what is offered 
and who is served? The CIP (Qassification of Instructional Programs) estabUshed eleven 
distinct two di?it codes, 128 four digit codes, and 567 six digit codes for classifying 
vocational education.* For purposes of manpower planning or assessing access to 
vocational education, defining participation at the two digit level has never been very 
satisfactory. The distinctions are not fine enough to be useful. At the other extreme, six 
digit reporting includes many rather esoteric programs with very few participants, 
nationally as well as at die state and local levels. Even four digit reporting far exceeds the 
number of programs typically offered in secondary and postsecondary institutions. What 
degree of detail, therefore, is appropriate? 

As long as transcripts are the basis for analysis, this question need not be answered 
a priori. With tiie transcripts, the analyst may impose whatever degree of specificity tiie 
data will support. If, however, die analyst does not have direct access to transcripts, the 
degree of programmatic specificity must be decided before data collection begins. For ex- 
ample, if local officials are expected to report counts of program completers based on ♦heir 



*Thc Classification of Instructional Programs has not been universally implemented. Many states continue 
to uss the old OfTice of Education (OE) codes. 
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own perasal of student transcript information, they must be told what kind of pmgrammatic 
detail is desired 

The appropriate degree of detail, unfortunately, cannot be consistent for all states 
and localities. If student records arc automated, then die least burdensome requirement 
may be six digit reporting, as it would require additional |irogramming to aggregate counts 
of students within two or four digit detail, Conputer-generated reports ftom fully auto- 
mated student data of participation by race and by sex are straightforwaixL The report 
would be based on information from only three variables — program code, race, and sex — a 
single task for any noodem database software. 

If student records are not autcxnated, or if automated transcript records carmot easily 
be linked to automated records on student characteristics, reporting becomes much more 
problematic. Under these conditions, a printed form must be completed manually. The 
form must contain all possible combinations of the three variables, which, depending on 
the number of values each variable may take, can result in an enonmous form. For exam- 
ple, maintaining the complete six digit detail would, therefore, require a form with 5,670 
separate cells (567 program codes x 5 racial/ethnic categories x 2 sex categories = 5,670). 
Even manually reporting at the four digit level would require a form with 1 ,280 ceUs. It is 
this kind of proliferation of data cells that so frustrated 0MB (Office of Management and 
Budget) in its efforts to minimize data burden from federal requests for information. 



What is Accomplished? 

Historically, the primary goal of vocational education has been to prepare students 
for employment in occupations requiring less than a baccalaureate degree. Although the 
major aim of vocational education has been preparation for employment, the primary test of 
the effectiveness of vocational education has been employment /:<?r se. Federal law tradi- 
tionally has sought to evaluate the effectiveness of vocational education programs in terms 
of placement in occupations related to training. Employer satisfaction with the skills and 
attitudes of completers from vocational education programs has been an added criterion in 
recent years. 

Assessing the accomplishments of vocational education, therefore, has relied pri- 
marily on follow-up of participants in vocational education after they have completed or left 




vocational education programs. VEDS. for example, required states to determine the status 
of vocational education participants six months after having completed or having left a vo- 
cational education program. States were to report, by program, the number of students 
employed in occupations related to training, employed in other occupations, pursuing fur- 
ther education or training, enlisted in the military, or not in the labor force. 

This approach to foUow-up suffered from a number of serious deficiencies. First, 
response rates to the mail surveys were very low. Response rates of twenty-five percent or 
less were typical, despite efforts at mail and telephone foUow-up of nonrespondents. Such 
low response rates rendered the foUow-up data useless. Second, even were it possible to 
achieve acceptable response rates, the one-time foUow-up only six months after completing 
or leaving a program offered an incomplete description of the benefits (or lack thereof) of 
vocational education. Unlike employment training, which is more oriented to achieving 
short-term benefits, vocational education purports to contribute to longer-range employ- 
ment skills, a claim that could not be assessed with the one-shot, short-term foUow-up re- 
quirements under VEDS. Third, because the foUow-up data was aggregated first by pro- 
gram and then by state, the data could not be used to analyze variability in employment out- 
comes. Outcomes could not be linked to student or program characteristics to detennine 
why some participants or some pn)grams were more successful than others. Similarly, 
tiiere were no provisions for controlling the vagaries of the labor market itself and other 
external factors affecting outconoes. 

Even if such shortcomings could be overcome— and the national longitudinal stud- 
ies do pennit much better analysis of the long-nin benefits of participation in vocational ed- 
ucation—a major pn)blem would remain: the single-minded focus on employment out- 
comes is inappropriate. WUe labor market outcomes are a necessary indicator of the ef- 
fectiveness of vocational education, they are not sufficient. Many factors other than the 
quality of a particular program affect a student's employment histo /. Economic condi- 
tions. family fonnation. personal interests, motivation, and alternative opportunities all fig- 
ure into what students do after participating in vocational education. Under the right eco- 
nomic conditions, for example, graduates of very poor programs may all do very well in 
the labor market, at least for a few years. 

What is needed, therefore, in addition to labor market outcomes, are more direct 
indicators of the accomplishments of vocational education, particularly of the educational 
outcomes of vocational education. To date, other than infonnation on the number of 
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program completers, data on educational outeomes has been absent Although vocational 
education in many states has claimed to have become increasingly competency-based, virtu- 
ally no effort has been made to develop valid indicators of the nature and degree of compc- 
tency achieved by the instruction. Nor have there been any efforts to develop valid mea- 
sures of "value added" dimensions of this instruction— tl;e gains and value added tiiereby 
from basic and job specific skills realized by students from participation in vocational edu- 
cation. Similaily, while vocaticmal educators have long maintained tiiat tiie applied, hands- 
on approach of vocational education provides an effective alternative to conventional aca- 
demic courses for imparting basic skills in reading, writing, mathematics, and science, 
there is lilde evidence of developmental work to produce indicators showing whether these 
gains have, in fact, occurred None of this should be surprising or, perhaps, even a cause 
for apprehension. American public expenditures in die social sciences, humanities, and 
even in engineering are far higher tiian in vocaticmal education. Yet these fields have begun 
a search for indicators of educational accomplishment in terms of competence, values, or 
intellectual skills. 

It will probably be argued that measures of educational outcomes on a national scale 
would prove hopelessly burdensome. Such an argument would certainly be true if federal 
law sought detailed data on a uriversal basis. There are, however, alternatives to universal 
collection. For example, competency measures could be added to the testing components 
of the national longitudinal studies. Oversampling students in selected vocational education 
programs could be done to produce sufficient cell sizes for analyzing tiie achievement gains 
of program participants. 

Similarly, policy could seek to use general indicators of educational outcomes, 
rather than more detailed test results. For example, suppose federal policy required states 
to adopt the following generic definition of program completion: demonstrated acquisition 
of the basic and job specific skills necessary to perform successfully in an occupation re- 
lated to training. States and localities might be left free to (fetermine, in collaboration with 
employers, the precise mix of skills and levels of competency necessary for successful per- 
formance. However, whatever mix of skills and levels of proficiency were established, 
students would be required to demonstrate, tiirough competency testing or other means, the 
achievement of these requirements before being certified as having completed a particular 
vocational education program. As a general iriicator of educational outcomes, then, states 




and localities could be required to repon the number of completers of vocational education 
programs. We shall have more to say on this possibility below. 

Leaving states ftec to determine the mix of skills and required levels of proficiency 
will result, some will object, in nonuniform definitions of completion across states and lo- 
calities. Completing auto mechanics in CMfomia, tiierefore, will not mean tiie same thing 
as completing auto mechanics in Michigan. This lack of comparability will, some will ar- 
gue, prevent national aggregates of tiie data and make tiie data useless for purposes of na- 
tional policy. Here, tiien, is tiie place for a brief conmwnt on tiie quest for uniform 
definitions. 

Adopting universal, uniform dcf-^itions for vocational education is a longstanding 
hope in federal policy. The 1976 Amendments, as well as tiie Perkins Act, explicitly call 
for uniform definitions in tiie design of federal vocational education data systems. Unless 
carefully conceived, however, uniformity may be counterproductive. Consider tiie case of 
completion* If by "uniform" one means that conq)letion represents the demonstrated ac- 
quisition of the basic and job specific skills nccessaiy to perform effectively in a job related 
to training, uniformity serves a useful policy purpose. If, however, uniformity also means 
that completion represents acquisitiai of the same set of sldlls via the same means (e.g., a 
specific sequence of courses), insistence on uniforaiity may produce undesirable outcomes. 
Completing an auto mechanics program for employment in a rural area, for example, may 
require a very different set of skills than completion for employment in the more specialized 
labor markets of a larger metropolitan area. 

In short, vocational education programs are likely to be more effective if they are 
designed to meet local and regional labor market needs. As these needs may often vary, 
programs often will be effective only if they are not uniform in content or in the specifica- 
tion of skills required to perform in an entry level position. Policy should, therefore, seek 
uniform generic outcomes — for example, completion should uniforaaly represent readiness 
to work— but be wary of insisting on uniform standards for deteimining whether such out- 
comes have been achieved. 

Similarly, uniform standards for defining a particular state can be misleading. For 
example, a definition of the economically disadvantaged based on uniform income 
standards will not uniformly measure economic disadvantage at all. Such a definition will 
understate the degree of disadvantage in high-cost areas of the country and overstate it in 
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low-cost areas. The appearance of uniformity will have been achieved at the expense of 
real meaning. 

Leaving states and Realities to their own devices, of course, presents a different set 
of problems. The argument advanced here is not that wiDy nilly state and local definitions 
arc preferable to federal definitions. Rather, we arc simply cautioning that uniformity in 
definiticMis must be q>proached with care and flexibility. 

In summary, what is accon[q)lished in vocational education needs to be more 
broadly conceived While labor market outcomes should continue to be monitored, they 
should not be the sole criteria by which die effectiveness of vocational education programs 
are assessed Educational outcomes arc, in many rcspects, more witfiin die control of vo- 
cational educators and, in this respect, morc appropriate measurcs for accountability. They 
have the added advantage of often being easier to measure. 



What Does Vocational Education Cost? 

Congress has asked a number of questions about the finances of vocational educa- 
tion. These include the following: 

• How much state and local money is expended for vocational education? 

• How much is spent on vocational education for students with special needs? 

• What arc the "excess costs" required to improve the access of students with 
special needs and for what are the costs incurred? 

• What are the relative costs of different programs and different delivery systems? 

• How is federal money allocated among eligible recipients? 

• What do federal funds buy? 

Some of uhese questions arc easier to answer than others. Although determining 
what is spent for vocational education may seem straightforward, it is, in fact, quite 
difficult. At the state level, most state education agencies (SEAs) can easily keep track of 
the allocation of federal vocational education funds to eligible recipients, but wiih the 
exception of those few states that provide separate state funding for vocational education, 
most states cannot determine how much state nx)ney is spent for vocational education. 
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State funds for vocational education are :illocated as part of general assistance to local 
educatiai agencies (LEAs), and, consequently, cannot be traced to a particular part of the 
cuniculum. 

Similarly, at the local level, LEAs maintain accounts that distinguish revenue by 
source (federal, state, and local) and expenditures by object (e.g., salaries, benefits, sup- 
plies, and equipment). Additionally, they can generally distinguish expenditures for in- 
struction from administration or counseling. Most LEAs, however, do not maintain ac- 
counts of expenditures by instructional program. Therefore, they do not know how much 
state and local nx)ney is spent for vocatimal educatim. Estimates can be and are made, but 
these are subject to cmsidcrable error and variability over time, as evidenced by the rvidely 
fluctuating estimates produced by VEDS and its predecessors. 

If it is difficult to determine how much is spent for vocational education as a whole, 
then it is even nxnt difficult to determine how much is spent by each type of vocational ed- 
ucation progranL VEDS sought expenditures by two digit program code, but the informa- 
tion submitted was highly suspect, often estimated simply on the basis of the distribution of 
enrollment among the different programs. Even if this prorating technique were an accept- 
able procedure for estimating expenditures by program— ^d this is not likely— die under- 
lying enrolhnent data was inaccurate and therefore could not possibly lead to accurate es- 
timates of expenditure by program. 

Even if accurate estimates of expenditures could be obtained by two digit code, the 
utility of such information is by no means clear. The two digit distinctions are too general 
to permit any useful analysis of differences in costs and expenditures among programs. 
Analysis at die four or six digit program level, which couH be useful for resource alloca- 
tion policy, is simply beyond die capabilities of most LEAs and SEAs on a regular basis. 

A related problem is the calculation of "excess costs" associated with providing 
services to students with special needs. Funds allocated under die setasides for handi- 
capped and disadvantaged students, in both the 1976 Amendments and the Peridns Act, arc 
to be used to pay for not more than one-half of the funds expended above the average ex- 
penditures per student for students with special needs. In practice, most LEAs have found 
it impossible to comply with this requirement. Not only do they not know precisely what 
average expenditures per student are for vocational education, but also dieir accounting 
systems are not designed to keep track of additional expenditures on certain types of 
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students or prognuns. To satisfy the fedend law, therefore, LEAs ait likely to use setaside 
funds only for items that cleariy would never be purchased as part of a regular program. 
The requirement, therefore, encourages compliance behavior, possibly at the expense of 
more effective but less traceable alternative uses for the funds. 

FmaUy, it should be noted that while it is easy to tnu^e the aUocation of fede^ 
funds to an LEA, it is difficult to determine how federal funds alone are spent because it is 
difficult to avoid conmungling with state and local revenues. Even separate accounting for 
federal revenues, a standard local practice, does not really sdve this problem because of the 
"substitution effect" For example, suppose federal funds are used to buy computing 
equipment One has no way of knowing whether, in the absence of federal funds, the local 
recipient would have bought the computing equipment with state and local dollars. If the 
equipment would have been purchased with state and local dollars and federal dollars sim- 
ply substituted for these funds, then the federal funds did not enable the purchase of the 
computing equipment but rather enabled the purchase of whatever was bought with the 
state and local dollars that otherwise would have been spent on the computing equipment. 
Sorting out these Mbstitution effects and establishing a direct causal link between the recip- 
ient of federal funas and expenditure for a particular purpose is very difficult. Therefore, 
attempting to isolate the effects of federal expenditures may not always prove firuitful. 

To summarize, obtaining good data on vocational education has been frustrated by a 
wide variety of misconceptions and misunderstandings about how vocational education op- 
erates across the fifty states and territories. LI!:* secondary and postsecondary education, 
vocational education is highly decentralized and subject to substantial variation in what is 
offered and how it is delivered Moreover, Congress has asked questions of vocational 
education that have not been asked of other parts of the education system. It would prove 
as difficult to determine what the nation spends on mathematics and science educaticm as it 
has been to calculate what is spent on vocational education. The question has never been 
asked, however, of mathematics and science. Similarly, if one were to require a universal 
follow-up of participants in the academic curriculum six months after high school gradua- 
tion or completion of a postsecondary degree, ; .sponse rates probably would be no better 
than those obtained for completers of vocational education programs. Unfortunately, be- 
cause such information has been sought only for vocational education, it has been tempting 
to assume that the failure to obtain good data is son^how unique to vocational education— 
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a failure of the enteiprise rather than a failure of those asking the questions or devising 
strategies for answering them. 

Despite these difficulties, a great deal has been learned about gathering infmnaticm 
on vocational education. As reauthorization approaches and consideration is given once 
again to data requirements, it is essential that what has been learned be carefully consid- 
ered The following section seeks to use what we know to sU. jh the outlines of future 
data collection efforts. 



EFFECTIVE STRATEGIES FOR COLLECTING DATA 
ON VOCATIONAL EDUCATION 

In thinking about how best to collect data on vocatk)nal education, it is important to 
distinguish between two types of data needs: data for accountability and data for policy. 
One strategy will not serve both purposes. The data necessary to ensure that recipients of 
federal funds are conforming to the official requirements of the law needs to be maintained 
annually for all recipients. For purposes of accountability, sampling will not usually suf* 
fice; each recipient needs to maintain sufHcient, auditable information. Consequently, be- 
cause these data requirements must be imposed universally, they should be simple and 
minimize data burden- Moreover, as long as the information is maintained by the recipient 
for use in audits or for review by the interested public, it is not essential that all data needed 
for accountability be reported to a central federal agency. 

In contrast, assessing federal policy options for vocational education requires dif- 
ferent kinds of information. Data requirements are considerably more detailed. Moreover, 
it will often be useful to follow the same students, staff, or institutions over periods of 
time. While these data requirements are more complex, the information is not needed as 
frequently, nor is it needed from the universe of providers of vocational education. As 
long as information is available to inform the five-year reauthorization cycles of federal 
legislation ant long as it presents a nationally representative picture of the condition of 
vocational education, it is not necessary to impose annual data collection requirements on 
everyone. Less frequent, national samples will suffice. Furthermore, national samples are 
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better suited for collection by a single agent (e.g., a federal agency or a designated contrac- 
tor) and are better able to use unifonn definitions and unifonn data collection strategies. 

To illustrate this distinction between data for accountability and data for policy, 
consider the issue of what constitutes vocational education and who should be counted as 
participating in it VEDS took the position that only students enrolled in programs ap- 
proved under the State Plan* and therefore eligible for federal funds* were to be counted for 
federal reporting. For purposes of accountability, this position was eminenUy sensible 
since Congress wanted to know how many students direcUy benefited from federal fands. 
Moreover, requiring recipients of federal funds to determine and report the number of stu- 
dents served is a longstanding requirement of program managersent 

For policy purposes, however, this definition of vocational education leaves much 
to be desired. As long as states enjoy considerable discretion in determining what pro- 
grams are eligible for federal funds, the "State Plan universe" wiD fall far short of describ- 
ing the total vocational enterprise. To assess the extent of coverage achieved by federal 
funds, to evaluate the adequacy of the labor supplied by various training institutions, or to 
assess the relationship between vocational education and other aspects of the secondary and 
postsecondary system requires information that describes with reasonable accuracy the na- 
ture of the overall enterprise. 

The discussion that follows concentrates primarily on data requirements for policy. 
Data required for accountability would depend on the precise features of federal policy, 
which will change over time. Data requirements for policy, however, are more stable and 
therefore can be incorporated into the ongoing data collection efforts of the federal 
government. Five major areas of data needs are examined: governance and delivery 
systems, student information, staffing, finance, and facilities and equipment. In describing 
infOTmation to be collected in each area, we do not claim to be exhaustive; rather, we have 
sought to identify what we consider to be the minimum data requirements for each area. 

Governance and Delivery Systems 

Federal law has required states to designate a "State Board,'' operating as the "sole 
state agency," to receive and oversee the distribution and uses of federal funds for voca- 
tional education. While this requirement achieves a final point of accountability for federal 
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funds, it seldom reflects the true organizational structure of vocational education in the 
states. For exanq)le, in numy states, separate boards govern secondary and postsecondary 
education, and each board has responsibility for vocational educatioi at its respective 
levels. Similaily, many states operate systems of area vocational schools or vocational 
technical institutes. Area vocational schools typically serve both secondary and adult stu- 
dents. In some states, these area schools are under the direct control of the secondary gov- 
ernance system; in other states, they are governed by their own boards. A few states have 
established separate state boards for vocaticmal education that operate independendy of the 
secondary and postsecondary systems. 

Complete, current infonnation on governance and delivery systems has not been 
collected since the National Study of Vocational T'lucation Systems and Facilities by 
Woodruff in 1978. Federal policy deliberations, as wt.l as odicr ongoing data collection 
efforts, would benefit from a periodic updating of this study. The following information 
should be collected or updated at least (mce every five years: 

• the primary boards responsible for secondary and postsecondary vocational 
education; 

• the constitution and govemkg authority of each state board; 

• the administrative agencies for each board, including the administrator with 
overall responsibility for vocati(Mial educati(Mi; 

• the types of institutions (e.g., comprehensive high schools, area schools, 
community colleges, technical institutes) and the number of each type 
adminisrered by each board and their sources of funding; 

• the board designated for purposes of federal policy to serve as the State Board 
of Vocational Education (assuming this federal requirement is maintained); 

• the responsibility of each board for secondary, postsecondary, and adult 
vocational education; and 

• the formal provisions for interaction and coordination among state boards and 
state agencies, including any legislatively created coordinating authorities. 



24 

CO 



Student Information 

Student information falls into two broad categories: program participation 
(enrollment) and follow-up- In bodi cases, there are needs for relatively shon-term 
information diat can be collected in die course of each reauthorization cycle, as well as 
Icxiger-term needs that cannot be carried out widiin five years (e*g*, die interacticHi between 
secondaiy and postsecondaiy participation patterns in vocational education). 

Program Participation 

By far the best source of information to date on the enrollment of secondary and 
postsecondary students in vocational education is die student transcript. However, no na- 
tional transcript study has yet been designed ftom beginning to end with the information 
needs of vocational education clearly in mind In every case, the study's utility for voca- 
tional education has simply been assumed, or addressed in midcoursc or after the fact. As 
a result, opportunities to make diesc transcript studies more useful for examining vocational 
education have been missed 

For purposes of evaluating federal policy, one major transcript study every four 
years would probably suffice. Recent work performed for the National Assessment of 
Vocational Education (NAVE) indicates tiiat participation patterns are fairly stable over 
time. Moreover, policies aimed at altering participation patterns are likely to take at least 
four years to generate discernible effects. 

At the secondary level, die National Assessment of Education Progress (NAEP) 
probably offers the best vehicle for a periodic transcript study conducted once every fom 
years. Widi proper planning for die 1990 NAEP data collection, transcript data could be 
easily merged with basic information on the characteristics of schools and students, includ- 
ing the students' NAEP test scores. Additionally, it has proven possible to add a special 
education supplement to NAEP to collect detailed information on the course-taking patterns 
of special education students. Designed in such a fashion, NAEP could supply die basic 
information Congress has sought on who is served and what is offered at the secondary 
level. Reporting participation by race, sex, academic and economic disadvantage, and 
handicapping condition would be straightfonvanL 

There is nothing equivalent to NAEP at die postsecondaiy level. The Integrated 
Postsecondary Data System (IPEDS) presendy seeks information on enrollment by 




program, but the accuracy and consistency of this data remains to be seen. Postsccondaiy 
institutions are much more likely to have the automated student record systems that should 
make IPEDS reporting mon accurate, but there still may be serious problems caused by 
differences among institutions in the procedures used to assign students to programs. As 
IPEDS surveys the universe of all postsecondary institutions, a transcript supplement to 
IPEDS would prove quite burdensome, even if conducted only once every five years. 
However, a transcript supplenaent for a sample of IPEDS institotions is worth considering, 
especiaUy if IPEDS enrollment data is not satisfactory. Other alternatives for obtaining 
postsecondary enrollment data need to be explored. 

In addition to these periodic cross section transcript studies using NAEP and 
IPEDS, the transcript components of the national longimdinal studies should continue. 
While these longitudinal studies rely on smaller samples than NAEP or IPEDS, they are the 
only mechanisms for understanding relationships over time. For example, the longitudinal 
transcript studies permit analysis of how secondary course-taking patterns in vocational and 
academic education interact with and influence postsecondary choices in regards to insti- 
tution and course-taking. Additionally, they provide a richer source of data on students, 
parents, teachers, and schools. The National Center for Education Statistics (NCES) has 
begun a new longitudinal study about every eight years. Studies by the Department of 
Labor have been less frequent. 

As long as transcript data is available every tour years from other sources, the cur- 
rent pace of initiating a new longitudinal study every eight or ten years is adequate. These 
studies are very expensive. Moreover, as tfieir greatest value lies in generating information 
over extended time periods, one is simply forced to wait for results. 

While periodic transcript studies, supplemented with data from less frequent longi- 
tudinal studies, will supply most of the information needed on program partic nation, one 
major gap will remain: participation by students, mainly adults, for whom no transcript is 
maintained. The size of tfiis group is not precisely known, although there is considerable 
anecdotal evidence to suggest that it is substantial. Participation in proprietary schools, in 
customized training programs in community colleges, and in noncredit adult vocational pro- 
grams often does not produce a student transcript. Nor is there necessarily any information 
about the length of participation or what was acccnq)lished while enrolled. Determining 
what needs to be known about this kind of adult participation in vocational education and 
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how best to obtain it should be a major priority of any future effort to improve the quality 
of vocaticmal education data. 

A related area in which there is a need for better inforaaation is program completion. 
While transcripts will reveal the sequence of courses taken by a student and the grades 
obtained^ they will not always document that a particular program was convicted Lack of 
information on program completion is more a secondaiy than a postsecondary problem, as 
the postsecondary transcript will typically indicate the awaid of a degree or certificate. 
With transcripts, of course, one can simply impose standard definitions of what sequences 
of courses constitute progr*^. completion, but this approach can be quite arbitrary and fail 
to capture important local differences in completion requirements. Moreover, inconsisten- 
cies in names of course tides increases the arbitrariness of such a procedure. 

At the postsecondary level, IPEDS now annually collects data on completers of de- 
grees and certificates. There is, however, no secondary counterpart, and some considera- 
tion should be given to collecting such information, especially if it could be linked to other 
needs for more general information on numbers of high school graduates or numbers of 
students completing other types of course sequences (in mathematics and science, for ex- 
ample). Furthermore, regular collection of information on completers of secondary voca- 
tional education programs would assume more importance if completion data figured in ac- 
countability requirements. For example, basing federal funds' distribution on numbers of 
program completers rather than numbers enrolled would not only encourage secondary 
programs to be more results-oriented, but would also eliminate basing tfie allocation of 
funds on inaccurate and inconsistent enrollment data. Additionally, completion-based 
funds distribution would need to be carefully designed to avoid creating undesirable 
incentives to screen students inappropriately (Hoachlander, Choy, & Brown, 1989). 

If completion data is sought, more attention needs to be given to the definition of 
completion at botii die secondary and postsecondary levels. We have recommended else- 
where tiiat the secondary definition include high school graduation, as well as demonstrated 
acquisition of tfie skills needed to perform effectively in a job related to training 
(Hoachlander, Choy, & Lareau, 1985). At botii ^ft secondary and postsecondary levels, 
definitions tiiat stress die acquisition of skills may be more appropriate tiian those that are 
based on finishing a particular sequence of courses or accumulation of credits. 



27 



Student FoIIow-up Information 

For rigorous, mid- to long-temi follow-up on the impacts of participation in voca- 
tional education, the national longitudinal studies will remain the best sources of data. 
They cannot be topped for detail over time on student characteristics, education^ family 
formation, and employment history. The primary problems with the longitudinal studies 
stem from sample size* It is not possible, for example, to conduct follow-up for small to 
medium sized programs* Nor is it possible to make comparisons among states or types of 
delivery systems* Hence, some attention should be paid 20 opportunities forjudicious, se- 
lective expansion of these samples to study particular p/oblems of interest to vocational 
education. 

Sample size may also be expanded to accommodate the follow-up requiitments of 
states. In fact, several states contracted with NCES to expand the san9)le within their states 
so that they would have state specific information from High School & Beyond. 
Improving this kind of partnership between federal and state data collection should be 
encouraged 

Although accountability requirements may generate additional needs for shorter- 
term, more comprehensive follow*up data, it will probably be best to leave states 
considerable discretion in how they deHne, collect, and maintain this information. For 
policy purposes, the national longitudinal studies should remain the primary source of 
follow-up information. 

Staffing 

As the nation's teaching force ages and ?s enrollments grow slowly but steadily 
during the next decade, problems of teacher shortages are likely to intensify* Moreover, is- 
sues of teacher quality and representation of minorities are also likely to continue to be 
prominent on the education reform agenda* Until recently, the federal government collected 
little information on teachers. HEGIS supplied sonx information on postsecondary faculty 
and staff, but data on elementary and secondary education, including vocational education, 
has been lacking. 
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VEDS sought data on secondary and postsecondary vocational education staff by 
using two digit program codes. The data was requested by race and by sex. While tfiesc 
provided some very rudimentary data <m the distribution of staff among the broad program 
areas in vocational education, even if accurate, they were not very useful. The data could 
not be linked to other information on teacher qualifications, experience, and age that would 
have enabled a better understanding of the condition of teaching in vocational education. 
To the best of our knowledge, the staff data was never carefully analyzed for accuracy and 
consistency, nor, aside from die Congressional mandate tfiat staffing data be collected, was 
thoe any clear use for the inforamtion. 

More recentiy, NCES undertook the Schools and Staffing Survey of 1987-88. The 
survey is being conducted on behalf of NCES by die U.S. Bureau of die Census witii data 
expected to be available by December, 1989. TTiis integrated survey collects detailed in- 
formation on teachers, schools, and school districts. It also includes a private school ques- 
tionnaire. There is ample information on race, sex, age, teacher qualifications, experience, 
salary, household characteristics, and perceptions and attitudes toward teaching. 
Additionally, die survey is designed to permit follow-up at a later time to smdy patterns of 
mobility into and out of teaching. Wrile die survey will permit separate identification of 
teachers of vc^ational education, it will provide relatively littie detail within die field. 
Unfortunately, diis is a simple coding matter diat could have easily been addressed widi 
some more attention to die needs of vocational education prior to finalizing design of die 
survey instmment. Similarly, it would have been possible to include a few additional atti- 
tudinal and perceptional questions aimed more specifically at vocational education teachers. 

Despite diese shortcomings, die School and Staffing Survey provides an excellent 
model upon which to build vocational education staffing requirements for bodi secondary 
and postsecondary programs. Witfi some fine tuning and assurances diat die survey will be 
conducted with reasonable frequency, policy needs for staffing data should be met. Again, 
once e .ry five years should be sufficient for vocational education, perhaps with every 
odier survey undertaking a longer-term longitudinal component 
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Finance 



As long as problems of "substitution effects" are ignored, data on the distribution 
and uses of federal vocational education funds is readily available. Without exception, 
states know how they allocate their federal dollars to locals, and locals know how they 
spend the money. While occasional honor stories about misuse of funds can be expected, 
accounting for the federal money is not problematic. Procedures arc in place and reporting 
is largely satisfactory. 

Information on state and local spending for vocational education is far more prob- 
lematic. While estimates that die federal dollar constitutes about one in ten of state and local 
dollars for vocational education have been around for some time, the trutii is tiiat it could 
just as easily be one in fourteen or one in nine. As long as tiiis general order of magnitude 
is aU tfiat is needed for federal poUcy, Uus traditional baUpark estimate is probably suffi- 
cient, and collecting more accurate information on state and local spending could receive 
relatively low priority. 

On die otiier hand, if tfiere arc concerns about such issues as cost effectiveness, 
levels of support for relatively high-cost/high-tech programs, relationships between cost 
and quaUty and access to higher cost/higher quality programs, or die relative costs of dif- 
ferent types of delivery systems, better information on state and local expenditures for vo- 
cational education will be needed. Obtaining such information accurately is likely to pose a 
formidable task. 

If such information is desired for purposes of federal policy, die best approach for 
die next reauthorization cycle is most likely a carefully designed, one-time study of a 
rcpresentative sample of secondary and postsecondary institutions. To be done accurately, 
such a study would probably need to send teams of trained data collectors to die sampled 
sites to cull die necessary information from school records. As noted previously, rarely do 
secondary or postsecondary institutions maintain cost and expenditure data by program. 
Expecting diem to ferret out such information in response to a mail survey is certain to 
prove unworkable.^ 



Exceptions are most common in states that maintain separate vocational high schools, area vocational 
schools, or vocational technical institutes. Because ihese institutions are completely separate from other 
types of secondary and postsecondary vocational education, separate cost accounting and expenditure 
accounung is possible. These cases, however, are the exception rather than the rule. 
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In many respects, the lack of program cost data at the elementary and secondary 
levels is a sony comment on the status of financial management in the nation's school sys- 
tems. It would be nice to believe that vocati(»ud education could lead the way to adoption 
of universal program accounting— throughout all of education, not simply vocational edu- 
catioiv— or that the federal government could spearhead a broad-based effort for program 
accoundng. Neither scenario, however, is likely. At the secondary level, vocational edu- 
cation accounts for about one-fourth of all programmatic activity, and it is unlikely that the 
smaller portion of the secondary enteiprise could produce a full-scale reworking of state 
and local accounting systems. Similarly, the federal role in secondary and postsecondaiy 
educatim is too small to lead such a revolution. Consequently, until states and local 
institutions take up the challenge, a special study of vocational educati<Mi finance is likely to 
be the most effective means for obtaining useful informatirai on state and local finances. 



F.xilities and Equipment 

In the 1960s, many states used federal Vocational Education Act funds to ccmstrurt 
and equip new facilities for vocational education. While construction is no longer an 
objective of federal policy, program improvement objectives continue to allow equipment 
purchases with federal funds, and such purchases have proved widely popular. Not only 
are local schools generally strapped for state and local funds to purchase equipment, but 
also equipment purchases are easy to track for federal audit purposes. Additionally, local 
administrators are sometimes reluctant to use federal "unds for hiring staff because the 
uncertainty and instability of federal funds preclude long-term commitments to personnel. 

Since 1976, federal law has asked for information on facilities, but very little in- 
formation has been collected. The last national study was conducted oy Woodruff in 1978. 
HEGIS and IPEDS regularly collect some information on facility inventory at the postsec- 
ondary level, but there is no secondary information. Facilities and equipment are not a 
subject of NCES's School and Staffing Survey of 1987-88. 

That there has been no pressure for such information, despite the legislative " .i- 
guage, may indicate that national information <mi facilities and equipment is a relatively low 
priority. Precisely what information the federal government needs in this area, and why. 
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should receive some careful attention before any new data initiative is started. In any event, 
it is likely that whatever infonnation is collected will not be needed more frequerdy than 
once every five years. 



RECOMMENDATIONS FOR FEDERAL POLICY 

Meeting national needs for information on vocational education does not require 
creating a totally new data system. Most of the means are already in place. In some cases, 
these require further refinement to ensure that the data needs of vocational education are 
adequately covered. In other instances, issues of timing need to be addressed to ensure that 
the information is available when it is needed to inform federal policy. In a few cases, 
there are major information gaps that may be necessary to fill. Finances and facilities, 
especially at the secondary level, are two examples. 

However, even if there is agreement that some majw new data collection efforts are 
needed, careful consideration should be given to integrating the needs of vocational educa- 
tion into the needs of education in general. A study of program costs, for example, need 
not be limited to vocational education. Similarly, a study that looked at the condition of 
facilities and equipment for secondary education generally, in addition to some more spe- 
cific information on vocational education, may be more valuable than one simply limited to 
vocational education. By integrating the collection of vocational education data into efforts 
to collect information on the larger education systems of which it is a part, not only is vo- 
cational education more likely to enlist broader-based support for data collection, but also 
comparisons with the rest of the education system can be made more easily. It is much 
more useful, for example, to know what secondary vocational education programs cost 
relative to other tyi>es of secondary education than it is to know the cost of vocational edu- 
cation in isolation. An integrated strategy for data collection will help to ensure that the 
comparisons are made nwre consistendy and accurately. 

If these integrated attempts at data collection are to be successful, however, they 
must be better informed about tiie needs and operations of vocational education. To this 
end. Congress should consider tiie creation of a National Vocational Education Data 
Advisory Council (NVEDAC). NVEDAC would serve as a central, national forum for dis- 
cussing tiie needs for data in vocational education and coordinating tiiem witii other national 
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data collection efforts. It could also review instrumentation and tabulations of initial statis- 
tics. Membership in NVEDAC should include, but not necessarily be limited to, tiie 
following: 

• representatives of die National Center for Education Statistics responsible for 
secondary, postseccxidary, and longitudinal studies; 

• the Office of Vocational and Adult Education, U.S. Department of Education; 

• the Office of Special Education and Rehabilitative Services, U.S. Department of 
EducaticHi; 

• tile Office of Planning, Budgeting, and Evaluation, U.S. Department of 
Education; 

• the U.S. Bureau of tiie Census, U.S. Depamnent of Commerce; 

• die Department of Labor, 

• the Department of Defense; 

• the National Center for Research in Vocational Education; 

• the National Center for Education and Employment; 

• tfie Council of Chief State School Officers; 

• die American Association of Community and Junior CdUeges; 

• die American Vocational Association; 

• die National Association of State Directors for Vocational Education; 

• die National Advisory Council for Vocational Education; 

• die National Occupational Information Coordinating Committee; 

• state and local secondary and postsecondary administrators familiar not only 
with state and local data collection but also with state and local needs for 
information; and 

• education researchers and evaluators. 

Just as die quality of vocational education data can be improved by centralizing co- 
ordination of data collection strategies in a national council devoted to that purpose, so 
could die use of diis data be improved by lodging major responsibility for reporting on and 
analyzing these data in a central place. Today, it is often the case diat die data exist to 
answer a particular question, but it is not always immediately apparent who knows where 
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to find the infonnation, how to extract it, and how to present it in a fashicm that respects the 
statistical constraints on die precision of die data. 

The logical choice for dus responsibility is die National Center for Rec?iarch in 
Vocational Education (die Center), and Congress should formally charge die Center widi 
being a central repository for national data relevant to vocational education and widi 
responding to requests for information from Congress and die Executive Branch. Such a 
chf e would not mean, of course, diat die Center would be die only user of diis data or 
diat all requests for information must go dirough die Center. Ratiier, it would clearly 
establiJi diat die Center should be expected to maintain a conqmehensive kno'vledge about 
what national data are available on vocational education and to have die mediodological 
expertise to report this infomiation accurately. 

Note die emphasis on "repository." Under no circumstances should die Center be 
given die responsibility for collecting national data on vocational education. For die rea- 
sons ahcady outlined, collection of data on vocational education needs to be integrated widi 
ongoing, routine efforts to collect education data generally. Collection is best done by fed- 
eral agencies (e.g., NaticmaJ Center for Education Statis'ics; Office of Vocational and Adult 
Education; Office of Planning, Budgeting, and Evaluation; & the U.S. Bureau of the 
Census) diat not only maintain die necessary technical expertise but also are in die best 
position to evaluate the broader education policy perspectives. 

What should federal legislation say about what data are to be coUeacd? Witii the ex- 
ception of Section 423, which calls for die universal collection of enrollment of handi- 
capped students by four digit code, die language of die Perkins Act does a good job of 
specifying die kinds of data needed for national policy and permitting sufficient mediod- 
ological flexibility to obtain die information. Section 423 does need to be rewritten to 
permit sampling and less fiequent data collection. Whedier Congress really wants data on 
state and local expenditures, and to what end, needs clarification, as does the request for 
data on facilities. Additionally, most of die information is probably not needed more dian 
eveiy five years, widi different data collection efforts undertaken sequentially duoughout 
die five-year reaudiorization cycles. 

An additional problem widi existing law is die lack of any specified level of funds 
for collecting die information. Widi die federal investment in vocational education ap- 
proaching one-billion annually, it would be wise to ensure diat die information needed for 
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national policy in this area is adequately supported Precisely what this figure should be 
will require some investigation. Howevert given the costs of existing data collection ef- 
forts, it is quite likely that the costs of well-defined vocational education supplements 
%;'ould not exceed three- to four-million doUan annually over what is presentiy expended 
With some additicmal fiinds for analysis and reporting, as well as support for NVEDAC, it 
seems likely that the cost could easily be kept under five-million dollars annually, or less 
than half of one percent of total federal e7q)enditures for vocaticmal education. 

In conclusicxi, while the collection of vocational education dau has been beset by a 
number of false starts with unsatisfactory results, it is important to keep i.i mind that 
Congress has asked much more about vocational education than any other component of 
equal size in the secondary and postsecondary educational systems Questions that on the 
surface would seem straightforward have, in fact, been quite complex. Had they been 
asked of other parts of the secondary and postsecondary curriculum, they would have 
proven equally difficult to answer. It is important to remember, therefore, that the prob- 
lems that have plagued tht collection of data on vocational education are not unique to it; 
rather, they are endemic in the larger systems we use to organize and account for secondary 
and postsecondary education. 

The history of efforts to collect data on vocational education, while frustrating, is 
also instructive. There are strong indications mat we have learned a great deal about how to 
do it better and that this knowledge is transferrable to other aspects of the educational enter- 
prise. This is, of course, if we should choose tc mand of it information comparable to 
what has been asked of vocational education. 
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